DOCOHEHT BESOHE 



ED 088 733 



SO 006 978 



&0THOR 
TITLE 
FOB DATE 
NOTE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Glennan, Thomas K., Jr« 

Towards the Practice of Program Development. 

16 Jun 72 

9p. 

MF-$0.75 HC-$1.50 

Educational Development; Educational Programs; 
Educational Research; Federal Legislation; "^federal 
Programs; Models; Program Administration; '^'Program 
Design; 'V'Program Development; "•"Program Evaluation; 
"•"Program Improvement; Program Length; Program 
Planning; Social Problems; Staff Improvement 
"•"National Institute of Education; NIS; NIE 
Archives 



ABSTRACT 
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I have been asked to put down a few words on the nature of 

IS- program development and trace the evolution of our thinking about 

OO 

CO this notion. Let xne begin by recounting a bit of not too well 

O 

Q authenticated history which may help to put xcy interest in program 

UJ 

development into some prospective - 

The efforts initiated when the Poverty Program and other Great 
^ Society programs began were supposed to have been based upon research 

but in reality were not. The Head Start program was preceded by a 



few research and demonstration programs but there was little solid 



evidence concerning their effectiveness- The Community Action 
^\ Program was, of course, presaged by the "grey 'area" and the juvenile 

delinquency programs funded by the Ford Foundation and HEW in the 
^ early 60's. The manpower programs seem more likely to have originated 

as part of these grey area programs or as part of the Area Redevelop- 
ment Act, which focussed on un- and under-employment in 
West Virginia and Appalachia generally. These programs were enlarged 
at an extraordinarily rapid rate. As a consequence, the best that 
could be done was to provide rather broad and superficial guidelines 
and instructions to practitioners in the field. 

A concomitant of the growth of Great Society programs was the 
growth of staff offices dealing with planning and evaluation. As 
« consequence these new social Initiatives were subjected to various 
sorts of evaluations dt both the local and natioaal levels earlier 
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in their history than is the case with most government programs. 
These evaluations were depressingly consistent in their findings. 
The impact of these programs seemed much lower than desired.,. The 
Head Start program seemed not to provide a cognitive head start, 
although it may well have been the vehicle for obta/ning better 
health care and social services for disadvantaged children* The 
Community Action Programs do not seem to have significantly 
redirected public resources towards the problems of the poor, although 
they provide employment for many and for a mechanism by which the 
poor can assert their concerns more effectively. And, compensatory 
education programs seem generally to be less than effective, with 
youngsters continuing to fail to achieve levels that have come to 
be viewed as the norms within society. 

The problem with most evaluations is that they cannot give us 
very much of a clue as to why the programs have failed* It could 
have been that program operations , particular ly when they were 
initiated under the hustle and bustle of the mid 60's simply did 
not Incorporate the practices suggested by the fragments of research 
that existed. The compensatory programs* curricula may have been 
used badly. The Head Start program may have not reflected the intent 
of its developers. 

Alternatively, the early program developers may not have used 
the appropriate theories. We may not have known what points of inter- 
vention were really appropriate. We surely do not know to what degree 
manpower programs fail because they are administered poorly or 
because they attack the wrong part of the problem, notably the 
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institutions of the labor market. Evaluations do not tell us this 
because they were too crude. They have looked only at outcomes. 
And, they couldn't lave done a great deal more because all of the 
multitude of factors affecting the outcome of social programs are 
confounded in normal operations so that it is difficult, if not 
impossible y to separate the causes of program success or failure. 

I suspect that the general public has interpreted the evalua*- 
tlons ' negative findings to mean that the types of treatment that 
have been used are not effective in dealing with poverty problems. 
Experts have not been quite so willing to do so, perhaps because 
they have a good deal of self-interest involved. They have argued 
that the programs that were evaluated did not adequately reflect the 
theories on which they were based , and that what was basically 
required was an improvement in the quality of these programs' operation. 
In order to test this, several major quasi-experimental, activities 
have been undertaken. The first was the Follow- Through program, fol- 
lowed by Planned-Variations experiment in the Head Start program. 
Another program, also called Planned-Variations, has been carried 
out in Model Cities and a few similar attempts have been made in 
compensatory education* The preliminary results from Follow-Through 
and Head Start also are not encouraging. Not only does there seem to 
be relatively little difference between the program types but there 
also seems to be relatively little difference in the relative status 
of persons participating in a program and those not receiving its 
services. This discouraging finding could weXl lead to the conclusion 
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that programs themselves are not productive and that perhaps the 
problem lies with the institutions into which they are thrust. For 
example » the labor market or normal elementary or secondary schools. 

Recently I have had some enlightening discussions with people who 
have developed programs that were subsequently incorporated into one 
of these Planned-Variations experiments. They argue that even after 
only two and three years in the Follow-Through and Head Start, the 
experimental projects within the schools no longer reflect the type 
of program as initially conceived. This is important because, in at 
least a few instances » the initial research indicated that disadvan- 
taged children gained significantly in competences that the program 
was designed to improve and these competences were sustained into the 
elementary school years by a substantial proportion of the program 
participants into the elementary school years. The contrast between 
f:he initially hopeful research and the actual experience of the 
^ Planned-Variations effort leads to one or two possible conclusions. 
One is that when programs are implemented in the local environment 
with the types of resources available there, they simply cannot come 
close enough to the original model to be effective. In other words, 
the original resear-.h program depended upon the highly trained 
competences of the researchers or the motivations of those involved 
in such research and that these competences and motivations are not 
consistently available on the local level* Alternatively, it is 
possible that the means for transferring the techniques and approaches 
inherent in research programs to the local operational program were 
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seriously deficient. If these deficiencies were remedied , it is 
possible that local people could in fact operate progratns that can 
improve the functioning of youngsters and sustain these improveinents 
through time. If this latter explanation is correct, then the 
Planned*Variations progratn may have failed really because it actually 
tested alternative models which had not been adequately transferred 
into operation. 

It is perhaps too early to determine whether or not this expla- 
nation of the failure of various types of intervention programs is 
an appropriate one, but it is surely worth paying more attention to. 
What such an explanation suggests is that we have paid entirely too 
little attention to what might be called the engineering aspects of 
program development: to developing training techniques, to providing 
technical assistance, and to providing materials that can be compre- 
hended and used by the kinds of person^ normally involved with 
program operation. We have paid too little attention to the types 
of people to be recruited to carry out a program activity and to the 
means by which one can install new curricula or methods into any 
functioning local system, such as the educational system. However 
pessimistic one might be about the efficacy of intervention programs, 
it is certainly true that no program, save perhaps a few of the 
scientific curricula programs, have involved extensive program 
development activities. 

What I am proposing is a new style of development activity which 
would have a much longer time frame than we have used over the parit: 
five CO eight years. 
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It would begin with a set of ideas which 
might be derived frotn exploratory and basic research, exemplary 
research and demonstration projects already in existence , or some form 
of program analysis, if a major effqrt to develop a new program or set 
of programs is desired, this research analysis material would be pulled 
together and a series of alternative program designs posited. 
Initially these designs, which would be fairly crude, would serve as 
a starting point for real program development activities* These 
activities might take between one and two years to accomplish and 
probably would proceed through a quasi-experimental framework utilizing 
control and experimental subjects. The activities would focus partially 
on development of materials and techniques, but also upon the develop- 
ment of the staff training procedures, selection criteria for staff, 
and strategies for intervention in local institutions if necessary. 
These techniques would be tested and refined in the second or pilot 
demonstration phasB of this process. 

Phase II would also be cast in an experimental framework, this 
time with alternative program designs being compared with one 
another. However, it would be expected that these projects would be 
highly developmental in nature with much experimenting with mechanisms 
for training and dissemination of program models, and with a highly 
skilled staff probably composed largely of people from the program 
developers own staff. The experimental framework would provide some 
discipline to the process and also allow a comparison in the pilot 
phase of program models with one another. At Che end of Phase II 



the mechanism intended to insure appropriate replicationswould be 
ready and the replication phase (Phase III) would begin. 

The replication phase might involve as many as four to six 
replications of each program model so that if there were^ for example, 
four alternative program models , one might expect between 12 and 16 
sites to be involved. IJie programs would be carried on over a long 
enough period of time that start-up problems would be taken care of. 
The experimental research design should be a good one, using random 
assignment of subjects. Participants in the process should be the 
kinds of people who would normally be hired at the local levels. The 
sponsors should be typical of those nonnaliy expected to sponsor such 
programs if the programs were initiated normally. Tlie involvement of 
the program developers in the activities of the local project should 
be only those that would be appropriate to the replication process 
(as designed in the earlier phase). In other words, program developers 
should not be extensively involved unless such involvement is part 
of the replication design (and expected to be part of any larger 
program activity). The intent of the replication phase is to get 
as close to a typical operational program using the program designs 
as little as is humanly possible. The evaluation might well, however, 
involve members of the program developers' staff in a process evalua- 
tion role so they can see the degree to which the intent of the 
program design is being followed in the operations. 

The key feature of this process is that the experiment is run 
with truly replicated tr.odcls ln53teu.: of rho cype of "hoc-hcu^e*' 
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Innovative models that might normally be expected in experiments as 
we have run them in the past. The process of replication should 
reflect that which is anticipated to be used in a national program or 
in support of local program operations if a national program is not 
contemplated. For example, if a dissemination is anticipated to be 
only in terms of written and other media, then that should be the 
manner in which the programs are initiated. If, on the other hand 
trainir*;; ir*3::i::u::c3 , tc:cluil»^al aiaiacauce, or other forms of oucsice 
assistance are contemplated, they would be used in the replication 
activities. 

This concept of program development requires some capabilities 
that do not readily exist either in the government or in the private 
sector. There are very few research organizations currently involved 
in program developinent that have the kind of staffs that would permit 
the extensive development of the programs and materials needed for 

i 

the I dissemination activities required here. The kinds of people 

t 

j 

required is substantially different than the research specialists 
who normally are involved in these activities and which the 
universities, for example, seek to attract. It is possible that some 
of these competences are available in private sector firms that have 
specialized in management training activities. That is something 
y€tp to be seen. 

\ Within the government there is a need to develop the capacity 
to carry on lengthy program efforts. Continuity of sorts Is required 

over a ::ive to savevi year p-jr^'lod. jince ^liis noc the sore of 
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continuity that has been widely available in the creative part: of 
the Federal structure, considerable attention should be paid to 
this problem. 

There is also a need for a kind of long range planning. The 
style of replication effort which is to be a part of the first two 
phases of this process will depend very much upon the projections as 
to how the program Tr^terials will ultimately be used. If we go to 
the highly decentralized type of social program aecis ionaLctkiug 
implicit in revenue sharing, the replication effort that is used 
should be appropriate for such a system. Prior to initiating any 
program development, attention should be given to this problem so 
that Phases I and II are properly constrained. 



